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Executive Summary 

This is the second and final report summarizing findings from an impact evaluation of a 
violence prevention intervention for middle schools. This report provides findings from the second 
and third years of the 3-year intervention. The U.S. Department of Education (ED) contracted with 
RTI International and its subcontractors, Pacific Institute for Research and Evaluation (PIRE) and 
Tanglewood Research, Inc., to conduct an evaluation of a hybrid intervention model that combines 
a curriculum-based program, Responding in Peaceful and Positive Ways (RiPP [Meyer and Northup 
2002a, 2002b, 2006]), and a whole-school approach, Best Behavior (Sprague and Golly 2005). The 
combined intervention was administered over the course of 3 successive years. Using a randomized 
control trial design (with entire schools randomly assigned either to receive the intervention or not), 
the evaluation assessed the intervention’s effects on both the full sample of students as well as on 
students at high risk for committing violence. Tanglewood Research provided implementation 
oversight along with site-based liaisons and coordinated training and technical assistance for staff in 
intervention schools. The developers of the two programs that constitute the intervention — 
Prevention Opportunities and University of Oregon — provided the program materials and 
conducted staff training. An earlier report presented findings after 1 year of program 
implementation (Silvia et al. 2010). 

Study Background 

ED’s Office of Safe and Drug-Free Schools (OSDFS) administers a variety of state and 
national programs under the Safe and Drug-Free Schools and Communities Act (SDFSCA, Title 
IV-A) that are focused on efforts to develop and maintain safe, disciplined, and drug-free schools. 
Drug and violence prevention activities under these programs are carried out in elementary and 
secondary schools, as well as in institutions of higher education. While there is now a lengthy set of 
school-based drug prevention curricula that have been evaluated using rigorous designs, much less 
evidence is available concerning effective violence prevention strategies in school settings. 

The need for evidence-based violence prevention programs is particularly critical for middle 
schools, whose students experience the highest rate of school-based violence relative to students in 
other grades. The most recent data available from the National Crime Victimization Survey show 
that in the 2006—07 school year, 4.3 percent of students aged 12 through 18 reported that they had 
been victims of a crime at school (DeVoe, Bauer, and Hill 2010). Data also indicate that the rate of 
victimization in 2007 for nonfatal violent crimes at school for students aged 12 through 14 was 
67 incidents per 1,000 students, compared with the rate for students aged 15 through 18, which was 
49 incidents per 1,000 students (Dinkes, Kemp, and Baum 2009). Students were also more likely to 
experience a violent event in middle schools (41 per 1,000) than in elementary (26 per 1,000) or 
secondary (22 per 1,000) schools. Data also indicate that bullying is a significant problem. In the 
2007—08 school year, 44 percent of middle schools, compared with 21 percent of primary schools 
and 22 percent of high schools, reported weekly or more frequent incidents of bullying (Dinkes, 
Kemp, and Baum 2009). 

Violence prevention strategies in schools can be divided into two broad types: curriculum- 
based programs and whole-school (or environmental) strategies. Curriculum-based programs are 
implemented in a classroom setting and typically aim to improve students’ social and problem- 
solving skills for dealing with conflict and managing violence. Whole-school (or environmental) 
approaches seek to influence the school environment through a variety of strategies, such as 
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increasing supervision of the school grounds, clarifying rules and consequences for student 
behavior, establishing reward systems to encourage positive behaviors, and training staff in 
classroom management. 

While evaluations of curriculum programs have yielded statistically significant results, their 
effect sizes are modest. A meta-analysis of school-based violence prevention evaluations from a mix 
of experimental and quasi-experimental designs reported an average effect size of 0.10 (Cohen’s d) 
for classroom-based social skills programs (Wilson and Lipsey 2005). RiPP (Meyer and Northup 
2002a, 2002b, 2006) is an example of such a curriculum and is one of two prevention programs 
selected for this study. RiPP has been subjected to three discrete evaluations by the program’s 
developers. One of these compared one classroom receiving RiPP to a nonmatched comparison 
classroom in the same school. In the second study, eight schools self-selected either to implement 
RiPP or be in the control condition. The third study used an experimental design. The two 
nonexperimental studies found that students who were exposed to RiPP reported significantly less 
physical aggression and lower levels of peer provocation than students in the comparison group but 
reported no significant differences in nonphysical aggression or drug use. Findings regarding self- 
reported delinquent behaviors were mixed (Farrell, Valois, and Meyer 2002; Farrell, Valois et al. 
2003). 

The single experimental evaluation of RiPP took place in 27 classes of 6th-graders in three 
urban middle schools. The evaluation found that after 1 year of exposure to RiPP, students reported 
fewer serious fight-related injuries and more participation in peer mediation compared with students 
in the control group but no difference on weapons-related violence or threats to teachers. School 
records showed fewer in-school suspensions and disciplinary violations for violent offenses for the 
students exposed to RiPP but no differences in out-of-school suspensions. The statistically 
significant outcomes were not maintained either 6 months or 1 year later (Farrell, Meyer, and White 
2001). In addition, intervention classrooms in two of the study schools were subsequently 
randomized either to receive an additional year of RiPP or not to receive an additional year. In this 
case, based on school records, there were fewer violent offenses for the RiPP group 1 year after the 
treatment group received 2 years of RiPP and the control group received 1 year of RiPP (Farrell, 
Meyer et al. 2003). 

A few school-based programs have sought to prevent violence by means of a whole-school 
approach such as the schoolwide Positive Behavior Support (PBS) approach (Sugai and Horner 
1994; Sprague, Sugai, and Walker 1998) and the whole-school component of Positive Behavioral 
Interventions and Supports (PBIS). Three evaluations of the whole-school approach examined 
changes in student outcomes without using any comparison group (Barrett, Bradshaw, and Lewis- 
Palmer 2008; Bradshaw, Mitchell, and Leaf 2010; Metzler et al. 2001), and a fourth used an 
experimental design in 63 elementary schools. The investigators from the latter study reported that 
the program led to an improvement in school staff members’ perceptions of school safety and an 
increase in the proportion of 3rd-graders meeting state reading assessment standards (Horner et al. 
2009). In addition, two evaluations have examined student outcomes after combining characteristics 
of both whole-school and curriculum-based approaches. One of these evaluations studied the 
approach using nonmatched, comparison schools (Sprague et al. 2001), and the other evaluation 
used an experimental design with a sample of eight schools (Orpinas et al. 2000). Neither study 
reported statistically significant differences on the targeted behavioral outcomes. 

The study’s research questions for impacts and implementation are provided in table ES-1. 
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Table ES-1. Research questions 



Intervention implementation 

• Is delivery of the violence prevention program consistent with its design and intended implementation? 

• What are the average costs of the program for participating schools and students? 

Student outcomes and impacts 

• Are there differences in the degree of violence and victimization in schools that implement the violence 
prevention program, relative to schools that do not implement it? 

• What is the impact of the violence prevention program on students who are at elevated risk for 
violence? 

• What are the outcomes of the violence prevention program for students with maximum program 

exposure or dosage? 



The Intervention 

Two research-based programs were selected through an open competition and advice from a 
panel of experts in the field of violence prevention: the RiPP program (Meyer and Northup 2002a, 
2002b, 2006) was chosen as the curriculum-based component of the intervention, and the Best 
Behavior program (Sprague and Golly 2005) (a formalized version of schoolwide PBS [Sugai and 
Horner 1994; Sprague, Sugai, and Walker 1998]) was selected as the whole-school component. 

These two approaches are considered complementary in that they target both individual- and 
school-level change mechanisms. RiPP is similar to other social skills programs in that it aims to 
increase social competence and improve problem-solving skills to reduce violence. While whole- 
school programs such as Best Behavior typically feature a schoolwide committee to review rules and 
policies for student behavior, Best Behavior (and PBS, upon which it is based) also includes a reward 
system to reinforce appropriate behavior. 

RiPP and Best Behavior were rated the highest by outside expert reviewers among a set of 
16 curricular and whole-school programs submitted for consideration by program developers of 
middle school violence prevention programs. Criteria for program selection included the program’s 
developmental appropriateness, overall quality of the approach and potential for reducing violence, 
theoretical foundation, and any outcome or process evaluation results, if available. The design of the 
intervention as a hybrid of the two types of programs was based on recommendations presented by 
a group of technical advisors in a design paper commissioned by ED prior to the current study (Bos, 
Weinstock, and Frankenberg 2004). The group’s recommendations were the result of discussions 
held during a series of technical working group meetings and were informed by several literature 
reviews and commissioned papers on the subject. Experts concluded that a combination of the two 
broad types of violence prevention strategies — curricular and whole-school approaches — offered 
the strongest potential for impacts. This conclusion was based on the experts’ judgment that the 
effect size was likely to be low for any one program, particularly a universal curriculum program, and 
that a whole-school strategy might boost that effect. 

While both RiPP and Best Behavior are implemented by school staff, in this study technical 
assistance was made available throughout the implementation period by on-site implementation 
liaisons trained and hired by the implementation subcontractor, Tanglewood Research. Liaisons 
(e.g., former school teachers or administrators) were hired to facilitate, coach, and monitor the 
progress and delivery of both programs. 
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The Curriculum: RiPP Program 

RiPP (Meyer and Northup 2002a, 2002b, 2006) is a universal, social-cognitive violence 
prevention program that focuses on the reduction of situational and relationship violence. The goal 
of the curriculum is to promote effective social-cognitive problem-solving skills; motivation and 
self-efficacy for using those skills; and school norms that support those attitudes and skills while 
reducing the appeal and perceived utility of violent behaviors and related attitudes. By targeting these 
attitudes and skills, the program is designed to increase social competence and thereby reduce 
violent behavior. 

The RiPP curriculum consists of 16 lessons (each lasting 50 minutes) per year in grades 6 
through 8. Each lesson builds on the previous lessons in a cumulative fashion. Similarly, each grade- 
level curriculum expands on the concepts taught in the previous year. In year one of program 
implementation, all students in the 6th through 8th grades received lessons designed for use in 
6th grade (RiPP-6) because all students were required to receive the foundational lessons before the 
more advanced RiPP materials. In the second year of program implementation, 6th-graders received 
the RiPP first-year curriculum, while the 7th- and 8th-graders both received the RiPP second-year 
curriculum. In the third year of the project, each grade received its grade-specific curriculum. Thus, 
by the end of the 3-year project, the study sample (6th-graders in the first year of the project) 
received all 3 years of the curriculum. 

The lessons in the RiPP program introduce and then reinforce the problem-solving model. 
The lessons comprise a variety of activities and strategies, including team building, social-cognitive 
problem solving, repetition and mental rehearsal, small group work, role playing, rehearsal of 
specific social skills for preventing violence, and didactic learning. Most lessons contain between 
four and six of these activities and are estimated to take between 5 minutes and 15 minutes per 
activity. Each activity is scripted and tied to a specific objective. Most lessons make use of the 
student workbook as part of the activities. 

The Whole-School Intervention: Best Behavior Program 

Best Behavior (Sprague and Golly 2005) provides a standardized staff development program 
that is designed to develop and administer effective school rules and discipline policies at both 
schoolwide and classroom levels to decrease school violence and antisocial behavior. The complete 
Best Behavior program is designed to be implemented on an approximately 3-year timeline, as 
individual school capacity dictates. The program is implemented by a school management team 
made up of teachers and administrators. Best Behavior involves intervention strategies at the school 
and classroom levels, including the following: 

• review and refinement of school discipline policies; 

• use of positive reinforcement and recognition for prosocial behaviors, both schoolwide 
and in individual classrooms; 

• clarification and teaching of behavioral expectations for student behaviors; and 

• systematic collection and review of patterns of discipline referrals to guide 
decisionmaking and planning. 

The Best Behavior management team is expected to create a systematic approach to 
developing schoolwide positive behavior supports. This approach includes four broad sets of 
activities, the first three of which should be implemented beginning in the first year. First, the team 
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is to conduct a schoolwide needs assessment to identify reasonable goals. Needs assessments are to 
be repeated annually. Second, the team is to define rules and expectations, with general rules (e.g., be 
safe, be respectful, be responsible) supported with expectations for all settings within the school 
environment. Rules and expectations are to be taught on a regular basis by all teachers and staff. 
Third, the team is to develop and support a positive behavior reinforcement system in which 
students are to be given token rewards for obeying rules and meeting expectations. Finally, in the 
second year of program implementation, the team is to develop a data-based decisionmaking process 
for identifying and addressing the needs of high-risk students. 

During annual training, Best Behavior prompts each school to develop its own specific 
strategies for achieving goals and objectives for each year while specifying the particular components 
that schools must implement. Best Behavior provides training on typical program implementation to 
enable each team to implement the various components specific to the needs of its individual 
school. For example, a sample lesson plan for teaching “be respectful” is provided, but teams are 
encouraged to adapt this prototype lesson to fit the language and unique cultural features of their 
school. As another example, all schools develop a reinforcement system, but the nature of the 
reinforcement system varies across schools (e.g., pizza parties vs. field trips). Therefore, while all 
schools implement each component of the program, the specific nature of the activities varies by 
school. 



Best Behavior was adapted to reinforce and complement the RiPP curriculum. As part of the 
integration with RiPP, Best Behavior recommended that a RiPP teacher be included on the school 
management team. In addition, Best Behavior trainings included a review of RiPP concepts and 
encouraged team members to think about ways of reinforcing these concepts schoolwide. RiPP 
lessons also reinforced Best Behavior concepts, such as the school rules. 

Evaluation Design 

The study was designed as a group-randomized control experiment in 40 middle schools 
serving grades 6 through 8. Twenty schools were randomly assigned to receive the combined 
intervention, and 20 schools were randomly assigned to serve as control schools (with no 
intervention beyond that which schools were already implementing). The intervention was delivered 
schoolwide by school staff trained annually by the program developers. Best Behavior was 
implemented by school management teams formed at each school, while RiPP was delivered by 
teachers in specific classes (e.g., science, social studies). School staff also received technical 
assistance and were monitored throughout implementation by trained site liaisons under the 
guidance of Tanglewood Research. 

The study was designed to estimate the impact of the RiPP and Best Behavior programs 
relative to what is being offered in the control schools. Most schools have ongoing violence 
prevention activities; therefore, the study compared the combined Best Behavior and RiPP 
intervention to extant school programs or the status quo. Schools that were implementing the Best 
Behavior or RiPP intervention (or a few other very similar programs listed on national registries as 
exemplary or model programs) prior to the study were excluded. 

Participating Schools 

Following recruitment of 13 districts and 40 schools, random assignment to condition was 
conducted within district, among pair-matched sets of schools based on the percentage of students 
receiving free or reduced-price lunches. The sites were geographically dispersed and represented a 
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range of district enrollment sizes. A majority of the districts were located in large urban or suburban 
areas with only three sites in rural districts. All participating schools were middle schools that 
included only grades 6 through 8. The average enrollment in these schools was 871 students and 
ranged between 462 students and 1,404 students. Minority students composed 65 percent of the 
student body, on average, and ranged between 15 percent and 100 percent. The average percentage 
of students receiving free or reduced-price lunches was 56 percent and ranged between 16 percent 
and 97 percent. There were no statistical differences between the intervention and control groups on 
these characteristics. Thirty-six schools (18 matched pairs) completed all 3 years of the project. For 
both pairs that did not complete all 3 years of the project, the control school was dropped from the 
study as a result of its paired intervention school ending its participation. The analytic sample for 
this report is comprised of data from these remaining 36 schools. 

Student Sample 

The analysis that estimated impacts after 3 years of the program used data from the full 
8th-grade student body, including those who entered the study schools as 6th-graders or 7th-graders 
and remained as 8th-graders and those who were new in spring 2009 (third follow-up). 

Nineteen percent of the students in the third follow-up sample were not in the original sample at 
baseline, while 81 percent were in both samples. These data were used to answer questions about the 
effects of the intervention across the general student population after 3 years of program 
implementation. Sixty-one percent (5,854) of students enrolled in the study schools completed a 
survey at the third follow-up that provided data for the impact analysis after 3 years of the program. 
Student demographic characteristics at baseline for the 36 schools that participated for all 3 years of 
program delivery indicate that minority students composed 72 percent of the sample in intervention 
schools and 61 percent of the sample in control schools. Forty-nine percent of the students in each 
group were male, and 60 percent in each group lived in single-adult households. A two-tailed /-test, 
obtained from multilevel regression models using the school as the unit of analysis, indicated that 
none of the mean demographic characteristics was statistically different between students attending 
intervention schools and those attending control schools. 

To address the research question regarding how the program impacts students at high risk 
for violent behaviors, we identified a subset of students at high risk for violent and aggressive 
behaviors, based on student responses to the fall 2006 baseline survey. Student self-report was 
considered critical for this study because many of the behaviors of concern often take place outside 
of the classroom and may not be reported to the school administrators. In addition, students have 
the best knowledge of their own behaviors and experiences. On the other hand, some students may 
hesitate to disclose information about their own behaviors, especially if they have concerns about 
the confidentiality of the data. The survey administration procedures used in this study were 
designed to make students comfortable with providing self-report data (see details in the “Data 
Collection and Outcome Measures” section below). A large study of the test-retest reliability of the 
Youth Risk Behavior Survey, which asks students about numerous risky behaviors, found that three 
of eight violence and victimization items had at least “substantial” reliability (kappa statistic 
>61 percent) and that five had at least “moderate” reliability (kappa statistic > 41 percent) (Brener 
et al. 2002). 

The high-risk sample was divided into two subgroups. High-risk perpetrators were defined as 
students who had self-reported perpetration of any one of eight serious acts of violence at school at 
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least once in the past 30 days. 1 Approximately 25 percent of the total treatment and control group 
sample, or 1,923 students, were assigned to the high-risk perpetrator group at baseline. High-risk 
nonperpetrators were defined as students who self-reported endorsing violence or reacting 
inappropriately to anger but who did not self-report any of eight serious acts of violence. 
Approximately 3 percent of the total treatment and control group sample, or 230 students, were 
assigned to the high-risk nonperpetrator group. 

Unlike the remainder of the students in the full sample, the high-risk samples were to be 
followed longitudinally; the research team attempted to survey the students in the high-risk 
subgroups even if they left one of the study schools and were attending another school in the same 
district at each follow-up. If a student went to a different district, the student was not followed. 
Seventy-one percent of the students identified as high risk at baseline in the 36 schools that 
participated for 3 years completed a survey at the third follow-up (N = 1,357). 

Teacher Sample 

Secondary outcome data were collected from teachers through an annual survey conducted 
in spring. This survey was administered to a random sample of 24 teachers (stratified by grade) at 
each of the intervention and control schools participating in the study; a new sample was selected 
each year. Eligible teachers included all full-time classroom teachers and could include RiPP teachers 
who delivered the curriculum in intervention schools. 2 Ninety-eight percent (N = 854) of the 
sampled teachers completed a survey in the third year. Among responding teachers in intervention 
schools, 30 percent were RiPP teachers. 

Data Collection and Outcome Measures 

Data for the study’s outcome measures to estimate intervention impacts were collected 
through student surveys in both intervention and control schools. Student surveys provided data to 
address the main impact research questions regarding school violence. Peer nomination measures by 
which data about individual students are simultaneously collected from many different peers were 
not considered due to the time-consuming and impractical nature of this type of collection within 
the context of a large multiyear effort. For this report, the baseline data collection for students 
occurred in fall 2006, and the follow-up to estimate impacts after 3 years occurred in spring 2009. 
Teacher data were collected through a survey administered to a random sample of 24 full-time 
teachers at each school in spring of each year to assess other program impacts besides main 
outcomes (school climate, victimization, feelings of safety). In addition to outcome data, the study 
team collected implementation data through the teacher survey, class records, annual school 
prevention coordinator and teacher interviews, and classroom observations. Annual interviews were 
conducted with the following: one person at each intervention and control school with the most 
knowledge about existing violence prevention activities, three randomly selected RiPP teachers at 
each intervention school to ask about teachers’ experiences with teaching RiPP, and three randomly 
selected members of the school management team at each intervention school to ask about their 
experiences with implementing Best Behavior. Finally, observations of RiPP sessions were 



1 The eight serious acts of violence were: (1) pick a fight with another student; (2) hurt another by hitting, pushing, slapping, or 
shoving; (3) throw something at another student to hurt them; (4) take, damage, or destroy someone else’s belongings; (5) try to force 
another student to do something they did not want to do; (6) threaten another student with a weapon such as a knife, gun, or club; 

(7) bring a weapon such as a knife, gun, or club to school; and (8) injure another student with a weapon such as a knife, gun, or club. 

2 ... . 

Schools identified a subject or subjects, such as health or social studies, through which RiPP could be delivered to all 6th-graders 
through 8th -graders. Teachers in those subjects were designated as RiPP teachers and trained to deliver the curriculum. 
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conducted by the evaluation team in three randomly selected classrooms (one classroom per grade 
level) in each intervention school. 

Student survey administration procedures were designed to address potential issues with 
reliability of self-report data on topics of a sensitive nature, such as violence and victimization. To 
safeguard the confidentiality of the students 5 data and encourage candid responses from students, 
the following measures were used: (1) prior to beginning the survey, students were read standard 
instructions to advise them how the survey should be completed and how the data would be used 
and safeguarded; (2) students were advised that the survey was voluntary and that they could skip 
any questions they did not wish to answer; (3) students responded to the survey by filling in circles 
for each item, so no handwritten responses by which students might be identified were required; 

(4) special labels with peel-away portions left only a bar code and no name on the survey booklets; 

(5) members of the evaluation team administered the survey, and no school staff were allowed to 
circulate among the students while the survey was in progress; (6) seating was arranged so that 
students had an empty seat between them, when possible; and (7) completed surveys were placed in 
large envelopes that were then sealed and taken from the school by the evaluation team. 

The primary outcomes are student violence and student victimization, both measured 
through student surveys. For each of these two outcomes, two additional subindices were created to 
better understand any differences between intervention and control schools with regard to specific 
types of violence. A second set of indices was created to examine possible secondary effects from 
the intervention, beyond the primary effects. These were: (1) student safety concerns; (2) teacher 
safety concerns; (3) teacher victimization; and (4) student prosocial behaviors. Finally, a third set of 
indices was created to examine possible intermediate effects from the intervention. The theoretical 
model for the combined intervention predicts that changes in these areas would precede changes on 
the primary outcomes and included: (1) student perceptions of behavior expectations; (2) student 
attitudes toward violence; and (3) student self-reported coping strategies. 

Analytic Approach for Estimating Program Impacts 

The study team evaluated program impacts using multiple regression models that predicted 
each outcome’s measure (e.g., violence, victimization) as a function of condition (intervention vs. 
control) and relevant covariates (e.g., demographic characteristics, school characteristics) using a 
mixed-effects regression model based on multilevel equations. Primary outcomes include self- 
reported counts of violent behavior and victimization occurring in the past 30 days. 

The full student sample and gender subgroup analyses used a matched nested cross-sectional 
model (matched analysis). Under this model, students are nested in schools; schools are nested in 
pairs and in experimental condition; and pairs are crossed with experimental condition (i.e., each pair 
is represented at each level of condition). The covariate models for students in the full sample 
predicted the average response at follow-up, adjusting for the following covariates: baseline school 
mean of the response, school size, and individual demographic variables (gender, race/ ethnicity, and 
number of parents in the household). For the gender subgroup analyses, the adjusted models 
included a gender-by-condition interaction effect. 

The statistical models employed to assess program outcomes among high-risk youth are 
different from those employed to assess program outcomes on the general population of students. 
For the high-risk youth, the interest is in whether or not the RiPP and Best Behavior intervention 
led to individual change across time. To address this question, nested cohort models using 
difference-in-difference estimation were developed to assess changes on self-reported measures of 
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aggression and victimization among high-risk youth in intervention schools relative to changes 
among high-risk youth in control schools, for both subpopulations of high-risk, nonperpetrator 
students and high-risk, perpetrator students. These models use data collected on the same sample of 
students at each measurement occasion. The repeated measures models for the high-risk subsamples 
contained the student’s treatment condition (intervention vs. control), data collection wave, wave- 
by-condition interaction effect, gender, race/ ethnicity, number of parents in household, and school 
size. Estimated program impacts reflect the net difference of the within-group change from pretest 
to third follow-up for intervention versus controls. 

To examine teacher outcomes, we employed multivariate models where teachers are nested 
within schools and schools are nested within matched pairs randomized to experimental condition. 
Hierarchical linear models account for the correlation of teachers within schools and for schools 
within matched pairs assigned to condition. The models predicted the average response at follow-up, 
adjusting for school size. 

Implementation Findings 

The key descriptive findings regarding implementation of the curriculum portion of the 
program across all years include the following: 

• In a majority of intervention schools, students were exposed to the full set of 

16 RiPP lessons in each of the 3 years of implementation. Between 61 percent and 
72 percent of schools delivered all 16 lessons to all classrooms in each year of program 
implementation, while another 17 percent to 22 percent of schools delivered all lessons 
in at least three-fourths of these classrooms. 

• The curriculum was not fully delivered with fidelity. In year three, teachers in 

44 percent of the intervention schools were observed to deliver lessons with few or no 
deviations from the written lesson plan (e.g., adding or modifying activities or changing 
the activity sequence), according to classroom observations by the evaluation team. With 
regard to teachers delivering lessons with few or no deviations from the prescribed 
teaching strategies (e.g., using role plays or small group discussions), teachers met this 
second criterion in 56 percent of schools in the third year. 

• Interviewed RiPP teachers cited challenges with using one or more of the 
prescribed teaching techniques or approaches. 3 Eighty-eight percent of teachers 
interviewed in year three mentioned difficulties with implementing at least one of five 
RiPP techniques or approaches, and 27 percent mentioned difficulties with 
implementing three or more of the five techniques or approaches. 

• The extent to which students were engaged with the curriculum declined by year 
three. The evaluation team observed the same cohort of students receiving the 
curriculum over 3 successive years. These students were found to be engaged during the 
lesson activities, exercises, and discussions in 89 percent of the intervention schools in 
year one and 69 percent in year three. 



3 

RiPP teachers were to use the following teaching techniques and approaches: make RiPP real (tie it to students’ daily lives); role 
plays; small group work, discussion, and brainstorming; encourage self-talk by students; use Review and Closure to begin and end 
sessions. 
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The key descriptive findings regarding the implementation of the whole-school portion of 
the program across the 3 years include the following: 

• Principal support and commitment for the whole-school portion of the 
intervention was mixed in year three. Principals at 72 percent of the intervention 
schools were rated as supportive in year three, according to liaisons who helped 
implement the program. In addition, liaisons reported that 50 percent of the principals in 
year three used their leadership to promote the program. In the third year of 
implementation, slightly more than half (56 percent) of the principals were regularly 
involved with the school management team. 

• By the end of the third year, the majority of intervention schools had instituted 
behavioral rules and rewards. In addition, a majority of teachers agreed that the 
rules were well defined and clear with regard to the behaviors being targeted. By 

the end of the third year, 83 percent of intervention schools had developed and posted 
school rules in the school, 78 percent had developed and instituted a token reward 
system for adhering to school rules, and 78 percent had developed lesson plans and 
taught the school rules in classrooms. In addition, 87 percent of teachers at intervention 
schools agreed or strongly agreed that school rules were clearly defined. However, a 
smaller percentage (64 percent) of the teachers surveyed in year three agreed or strongly 
agreed that that it was clear what consequences would follow when school rules were 
broken. 

• Among the cited challenges with implementing Best Behavior were finding time 
to implement the program, obtaining teacher buy-in, maintaining student 
interest, and funding the rewards program. School management team members 
interviewed in year three mentioned difficulties with finding the time to implement the 
program. 4 Others talked about issues with low teacher buy-in, a lack of student interest 
in the rewards offered through the reward system, and continued problems with funding 
the reward system. 

• By design, no control school implemented RiPP or Best Behavior during the 

3 years of the study. However, there were various violence prevention activities already 
in place in the participating schools. Between eight schools and nine schools in the 
intervention group and between six schools and seven schools in the control group 
administered classroom-based education other than RiPP across the 3 years. The types 
of programs implemented included gang resistance programs, character education 
programs, and individual presentations that were not part of a curriculum (most often, 
speakers, a video, or a lesson) focused on specific topics such as bullying, harassment, 
and dating violence. 



4 

Key program practices are as follows: (1) the school management team meets regularly; (2) rules of behavior are defined and 
adopted; (3) rules are posted throughout the school; (4) plans are developed for teaching rules; (5) rules and expectations are taught; 
(6) a reinforcement system is developed; (7) the reinforcement system is implemented school-wide; (8) discipline data are collected 
and reviewed; (9) focus is given to the needs of high-risk youth; and (10) a schoolwide needs assessment is conducted periodically. 
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Impact Findings 

These are the main findings regarding intervention impacts after 3 years. 

• There were no statistically significant differences between intervention and 
control schools on self-reported student violence or victimization (figure ES-1). 

On average, 8th-graders in the intervention schools reported engaging in 2.84 violent 
acts at school in the past 30 days, compared with 8th-graders in control schools, who 
reported engaging in 2.69 violent acts at school in the past 30 days. On average, 
8th-graders in intervention schools reported being victimized 4.14 times in the past 
30 days, while 8th-graders in control schools reported being victimized 4.16 times in the 
past 30 days. 

• There were no statistically significant program impacts on violence or 
victimization for students who were at risk for engaging in violence but who had 
not previously done so (figure ES-2). On average, 8th-graders in the intervention 
schools who were categorized as being at a high risk for violence but who had not self- 
reported any of eight serious acts of violence ever at baseline (nonperpetrators) reported 
at the third-year follow-up that they had engaged in 3.30 (change from baseline of 1.28) 
violent acts at school in the past 30 days. This is compared with high-risk, 
nonperpetrator 8th-graders in the control schools who reported at the third follow-up 
that they had engaged in 3.12 (change from baseline of 1.35) violent acts at school in the 
past 30 days. In addition, high-risk, nonperpetrator 8th-graders in the intervention 
schools reported being victimized an average of 4.30 (change from baseline of 3.60) 
times at school in the past 30 days. This is compared with high-risk, nonperpetrator 8th- 
graders in the control schools who reported being victimized an average of 3.79 (change 
from baseline of 3.41) times at school in the past 30 days. 

• There were no statistically significant program impacts on violence or 
victimization for high-risk students who had previously engaged in violence 
(figure ES-3). On average, high-risk, perpetrator 8th-graders in the intervention schools 
reported at the third follow-up that they had engaged in 3.95 (change from baseline of 
5.36) violent acts at school in the past 30 days. This is compared with high-risk, 
perpetrator 8th-graders in the control schools who reported at the third follow-up that 
they had engaged in 3.90 (change from baseline of 4.92) violent acts at school in the past 
30 days. Those in intervention schools reported being victimized 4.90 times at school in 
the past 30 days, on average (change from baseline of 6.29), while those in control 
schools reported an average victimization rate of 5.05 (change from baseline of 6.10). 

• There were no statistically significant impacts on either secondary or 
intermediate outcomes. In addition, after 3 years of exposure to the RiPP and Best 
Behavior intervention, student measures for secondary outcomes — including student 
safety concerns, teacher victimization and safety concerns, and student prosocial 
behaviors — did not differ between students in intervention schools and students in 
control schools. Also, there were no statistically significant differences on intermediate 
outcomes — that is, where the program logic model predicts change would be observed 
before it would be observed on the outcome measures. These include student self- 
reported coping strategies, student perceptions of behavior expectations, and student 
attitudes toward violence. 
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Figure ES-1. Main program impacts on self-reported violence and victimization, after 3 years of 
program delivery 
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NOTE: N= 2,784 students clustered in 18 intervention schools and 3,070 students clustered in 18 control schools. 
Generalized linear mixed models (SAS PROC GLIMMIX, Poisson distribution with log link function) were used to 
evaluate the program impact while accounting for the clustering of students within schools. Covariates in the model 
included the baseline school mean of the response variable, treatment condition (intervention vs. control), 
race/ethnicity, sex, number of parents in household, and school size. Program impacts were estimated as a model- 
adjusted event rate ratio (ERR) for intervention versus controls at follow-up, with 95 percent confidence limits. There 
are no statistically significant program impacts. 

SOURCE: Student survey, fall 2006 (baseline) and spring 2009 (third follow-up). 
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Figure ES-2. Main program impacts on self-reported violence and victimization, after 3 years of program delivery: High-risk, 
nonperpetrator subgroup (Via repeated measures) 
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NOTE: N = 70 students clustered in 18 intervention schools and 74 students clustered in 18 control schools. Generalized linear mixed models (SAS PROC 
GLIMMIX, Poisson distribution with log link function) were used to evaluate the program impact while accounting for the clustering of students within schools. 
Covariates in the model included gender, race/ethnicity, number of parents in household, and school size. Program impacts (with 95 percent confidence limits) 
were estimated via difference-in-difference models comparing change across time in the intervention versus control group. There are no statistically significant 
program impacts. 

SOURCE: Student surveys limited to a high-risk subgroup of 6th-graders surveyed in fall 2006 and followed up in spring 2009. 
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Figure ES-3. Main program impacts on self-reported violence and victimization, after 3 years of program delivery: High-risk, perpetrator 
subgroup (Via repeated measures) 
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NOTE: N = 552 students clustered in 18 intervention schools and 661 students clustered in 18 control schools. Generalized linear mixed models (SAS PROC 
GLIMMIX, Poisson distribution with log link function) were used to evaluate the program impact while accounting for the clustering of students within schools. 
Covariates in the model included gender, race/ethnicity, number of parents in household, and school size. Program impacts (with 95 percent confidence limits) 
were estimated via difference-in-difference models comparing change across time in the intervention versus control group. There are no statistically significant 
program impacts. 

SOURCE: Student surveys limited to a high-risk subgroup of 6th-graders surveyed in fall 2006 and followed up in spring 2009. 
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